Papers with Encoder-decoder models
Graph-based Filtering of Out-of-Vocabulary Words for Encoder-Decoder Models (P18-3)
Copied to clipboard
| Challenge: | Encoder-decoder models employ words that are frequently used in the training corpus but may still include noisy words. |
| Approach: | They propose a method for selecting more suitable words for learning encoders by utilizing co-occurrence information. |
| Outcome: | The proposed method outperforms the baseline method in Japanese-to-English translation and grammatical error correction tasks with an F-measure of 1.48 points higher. |
Whisper-UT: A Unified Translation Framework for Speech and Text (2025.emnlp-main)
Copied to clipboard
Cihan Xiao, Matthew Wiesner, Debashish Chakraborty, Reno Kriz, Keith Cunningham, Kenton Murray, Kevin Duh, Luis Tavarez-Arce, Paul McNamee, Sanjeev Khudanpur
| Challenge: | Encoder-decoder models have achieved remarkable success in speech and text tasks, but efficiently adapting them to diverse uni/multimodal scenarios remains a challenge. |
| Approach: | They propose a framework that leverages lightweight adapters to enable seamless adaptation across tasks. |
| Outcome: | The proposed framework improves speech translation performance through a 2-stage decoding strategy without requiring 3-way parallel data. |
Inflecting When There’s No Majority: Limitations of Encoder-Decoder Neural Networks as Cognitive Models for German Plurals (2020.acl-main)
Copied to clipboard
| Challenge: | Encoder-decoder models can be used to generalize to inflectional morphology and generalize new words, but they fail on tasks like German number inflection, where infrequent suffixes like /-s/ can still be productively generalized. |
| Approach: | They propose to use a dataset to collect data from German speakers to examine whether ED models can generalize the most frequently produced plural class. |
| Outcome: | The proposed model does not show human-like variability or ‘regular’ extension of other plural markers. |
Learning Neural Templates for Text Generation (D18-1)
Copied to clipboard
| Challenge: | Encoder-decoder models are uninterpretable and difficult to control in terms of content. |
| Approach: | They propose a neural generation system using a hidden semi-markov model which learns latent templates jointly with learning to generate. |
| Outcome: | The proposed model learns useful templates and achieves strong performance nearing that of encoder-decoder models. |
Exploiting Invertible Decoders for Unsupervised Sentence Representation Learning (P19-1)
Copied to clipboard
| Challenge: | Encoder-decoder models for unsupervised sentence representation learning discard decoder after training . decoded sentences are often used to make better predictions of words in a given sentence . |
| Approach: | They propose two types of decoding functions whose inverse can be easily derived without expensive inverse calculation. |
| Outcome: | The proposed models can learn good representations from encoders and decoders without expensive calculations. |
Masking in Multi-hop QA: An Analysis of How Language Models Perform with Context Permutation (2025.acl-long)
Copied to clipboard
| Challenge: | Multi-hop Question Answering (MHQA) adds layers of complexity to question answering tasks. |
| Approach: | They explore how LMs respond to multi-hop questions by permuting search results under various configurations. |
| Outcome: | The proposed model outperforms decoder-only models in MHQA tasks despite being significantly smaller in size . |